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COMPUTER ASSISTED SYSTEM AND METHOD FOR MINIMAL INVASIVE 
HBP, UNI KNEE AND TOTAL KNEE REPLACEMENT 

This application claims the benefit of US Provisional Patent Application No: 60/390,188, 
entitled "COMPUTER ASSISTED SYSTEM AND METHOD FOR MINIMAL 
INVASIVE HIP, UNI KNEE AND TOTAL KNEE REPLACEMENT", filed on June 21, 
2002. 

BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 

[0001] The present invention relates to a method and system for computer assisted 
medical surgery procedures, more specifically, the invention relates to a system which 
aids a surgeon in accurately positioning surgical instruments for performing surgical 
procedures, and also relates to reducing user interaction with the system for minimal 
invasive surgery. 

[0002] Many surgical procedures, particularly in the fields of orthopedic surgery 

and neurosurgery, involve the careful placement and manipulation of probes, cutting 
tools, drills and saws amongst a variety of surgical instruments. Computer-based surgical 
planning has been investigated by many researchers over the past decade and the promise 
of die technology is to provide better surgical results (with fewer procedures), decreased 
time in the operating room, lower resulting risk to the patient (increased precision of 
technique, decreased infection risk), and a lower cost. In image-guided surgery the vision 
of reality is enhanced using information from CT, MR and other medical imaging data. 
Certain instruments can be guided by these patient specific images if the patient's position 
on the operating table is aligned to this data. 

[0003] Preoperative 3D imaging may help to stratify patients into groups suitable 

for a minimally invasive approach or requiring open surgery. The objectives include the 
most accurate prediction possible, including the size and position of the prosthesis, the 
compensation of existing differences in leg lengths, recognizing possible intraoperative 
particularities of the intervention, reducing the operating time and the potential for 
unforeseen complications. 

[0004] Traditional surgical planning involves overlay of 2D templates onto planar 
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X-ray images, however this process is sensitive to errors in planar X-ray acquisition and 
magnification. Precise 3D models of implants superposed onto intra-operative calibrated 
fluoro is an improvement over current methods, however interpretation of these 3D 



models is not intuitive. 
[0005J In the case of X- ray imaging (fluoroscopy or CT scan), the surgical staff 

are required wear protective clothing, such as lead aprons during the procedure. Also, the 
imaging device must be present during the course of the surgery in case the patient's 
orientation is changed. This can be cumbersome and undesirable given the space 
requirements for such equipment, such as magnetic resonance imaging, X-ray imaging 
machine or ultrasound machine. Therefore, in such circumstances it is desirable to 
maintain the patient in a fixed position through the course of the surgical operation, 
which can prove to be very difficult Therefore, a surgeon has to be present for image 
acquisition and landmark identification. 

[0006] Image-guided surgery permits acquiring images of a patient whilst the 

surgery is taking place, align these images with high resolution 3D scans of the patient 
acquired preoperatively and to merge intraoperative images from multiple imaging 
modalities. Intraoperative MR images are acquired during surgery for the purpose of 
guiding the actions of the surgeon. The most valuable additional information from 
intraoperative MR is the ability for the surgeon to see beneath the surface of structures, 
enabling visualization of what is underneath what the surgeon can see directly. 
[0007] The advantages of 2D operation plarjning include simple routine 

diagnostics, as the X-ray is in 2 planes, simple data analysis, simple comparison/quality 
control on postoperative X-ray, and more beneficial cost-benefit relation. However, 2D 
operation planning module has the several drawbacks, it lacks capability of spatially 
imaging of anatomic structures, and implant size can only be determined by using 
standardized X-ray technology and has no coupling to navigation. The advantages of 3D 
include precise imaging of anatomical structures, precise determination of implant size, 
movement analysis of the joint possible, and coupling with navigation. However, 3D 
provides for more expensive diagnostics, as it involves X-ray imaging and CT/MRI 
imaging. Also, CT data analysis is time consuming and costly, and there is no routine 
comparison of 3D planning and OP result (post-op. CT on routine. 
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SUMMARY OF THE INVENTION 

(0008] In one of its aspects there is provided a computer-implemented method for 
enhancing interaction between a user and a surgical computer assisted system, the 
method includes the steps of tracking a user's hand gestures with respect to a reference 
point; registering a plurality of gesturally-based hand gestures and storing said gestures 
on a computer-readable medium; associating each of said plurality of gesturally-based 
hand gestures with a desired action; detecting a desired action by referencing said user's 
hand gestures stored on said computer-readable medium; and performing the desired 
action. 

[0009] In another one of its aspects there is provided a computer-implemented 

method for enhancing interaction between a user and a surgical computer assisted system, 
the method having the steps of: determining information for a surgical procedure from 
the orientation of a medical image whereby accuracy of said information is improved. 
The orientation of the medical image is obtained by tracking of the imaging device or by 
tracking of a fiducial object visible in the image. 

[0010] In another one of its aspects there is provided a method for a computer 

assisted surgery system, the method includes the steps of using 3D implant and 
instrument geometric models in combination with registered medical images, generating ■« 
2D projections of that instrument and/or implant, updating the 2D projection dynamically 
in real-time as the implant/instrument is moved about in 3D space. Advantageously, the 
dynamic 2D projection is more intuitive and provides ease of use a user. 
[001 1] In yet another aspect of the invention, there is provided a method for a 

computer assisted surgery system, the method having the steps of displaying a magnified 
virtual representation of a target instrument or implant size while smaller instruments or 
implants are being used. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0012] These and other features of the preferred embodiments of the invention will 
become more apparent in the following detailed description in which reference is made to 
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the appended drawings wherein: 

[0013] Figure 1 is a schematic representation of a computer assisted surgery 

system; 

10014] Figure 2 is a block diagram of a computing device used in the system of 

figure 1; 

10015] Figure 3 is a set of instruments for use with the system of Figure 1 ; 

[0016] Figure 4 is patient tracker for minimal invasive surgery; 

[0017] Figure 5 is a flow chart showing the sequential steps of using the system 

of figure 1. 

[0018] Figure 6 shows examples of landmarks defining a pelvic coordinate 

system; 

[0019] Figure 7 shows a way of calculating an anteversion or inclination angle; 

[0020] Figure 8 shows a virtual representation of a reamer; 

[0021] Figure 9 shows a femoral anteversion; 

[0022] Figure 10 shows guidance of a femoral stem length and an anteversion 

angle; and 

[0023] Figure 1 1 is a 2D projection of femoral stem model. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0024] Referring to Figure 1 , there is shown a computer assisted surgery system 10 
for performing open surgical procedures and minimal invasive surgical procedures on a 
patient 12 usually positioned horizontally on an operating table 14, Open surgical 
procedures include hip, knee and trauma surgeries, however computer assistance can 
facilitate minimal invasive approaches by providing valuable imaging information of 
normally hidden anatomy. Minimal invasive surgical procedures include keyhole 
approaches augmented by calibrated image information which reduce hospital stay and 
cost and greatly improve patient 12 morbidity and suffering. Such surgical procedures 
require a plurality of instruments 16, such as drills, saws and raspers. The system 10 
assists and guides a user 1 8, such as a medical practitioner, to perform surgical 
procedures, such as to place implants 20 using the instruments 16, by providing the user 
1 8 with positioning and orientation of the instruments 16 and implants 20 with relation to 
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the patient's 12 anatomical region of the operation, iuch as the hip area. 
[0025] As a general overview, the system 10 is used to assist the surgeon in 

performing an operation by acquiring and displaying an image of the patent. Subsequent 
movement of the patient and instruments is tracked and displayed on the image. Images 
of a selection of implants are stored by the system and may be called to be superimposed 
on the image. The surgical procedures may be planned using the images of the patient 
and instruments and implants and stored as a series of sequential tasks referred to defined 
datums, such as inclination or position. Gestures of the surgeon may be used in the 
planning stage to call the image of the instruments and in the procedure to increment the 
planned tasks, 

[0026] Referring to Figure 1, the system 10 includes an imaging device 22 for 

providing medical images 24, such as X-ray, fluoroscopic, computed tomography (CT), 
magnetic resonance imaging of the patient's 12 anatomical region of the operation and 
the relative location of the instruments 16 and implants 20. Generally, a C-arm, which 
provides X-ray and fluoroscopic images 24, is used as the imaging device 22. The C-arm 
can be positioned in the most convenient location for the procedure being carried out, 
while allowing the user 18, the maximum possible space in which to work so that the 
procedures can be freely executed. The C-arm 22 features movement about or along 
three axes, so that the patient 12 can be easily approached from any direction. The C-arm 
22 includes an X-ray source 21, an X-ray detector 23 and imaging software that converts 
the output of the detector into a format that can be imaged on display screen 25 for 
displaying the images 24 to the user 18. 

{0027J Radiation exposure is a necessary part of any procedure for obtaining an 

image to assist in calculating the proper angle of the instruments 16 and implants 20, 
however, radiation exposure is considered to be a hazard, an exposure to the user 18 as 
well as the patient 12 during orthopaedic procedures using fluoroscopy is a universal 
concern. Consequently, a reduction in the amount of radiation exposure is highly 
desirable. Typically, the images 24 are acquired during pre-planning and stored in a 
image memory 29 on a computing device 26 coupled to the C-arm 22. As will be 
explained further below, the acquired images 24 are referenced to a 3D coordinate 
framework. This may be done automatically by referencing the image 24 to the 
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fiameworkwhenacquiringtheimage24 or manually by formatting the image 24 to 
contain fiducials, either inherent from the imaged structure or added in the form of an 
opaque marker to permit registration between the images 24 and patients. Generally, 
the computing device 26 is contained within a housing and includes input/output 
interfaces such as graphical user interface display 28 and input means such as mouse and 
a keyboard. 

10028J To facilitate the performance of the operation, the position and orientation 

of the operative instruments 16 and implants 20 is displayed on the images 24 by 
monitoring the relative positions of the patient 12, instruments 16 and implants 20. For 
(his purpose, movement of the patient 12 is monitored by a plurality of positional sensors 
or patient trackers 30 as illustrated in Figure 4 attached to the patient 12 to report the 
location of orientation of the patient 12's anatomy in a 3-D space. One example of the 
position sensor is a passive optical sensor, by NDI Polaris, Waterloo, Ontario, that allows 
real-time tracking of its trackers in three-dimensional space using an infrared-based 
camera tracking 27. Therefore, the patient trackers 30 report these coordinates to an 
application program 32 of the computing device 26. Each patient tracker 30 is fixed 
relative to the operative site, and a plurality of patient trackers 30 are used to 
accommodate relative movement between various parts of the patient's 12 anatomy. For 
minimal invasive surgery, the patient trackers 30 used can have minimal access for 
attachment to the patient 12. 

(0029] To enable registration between the patient and the image 24 during the 

procedure, position sensors 32 are placed in distinctive patterns on the C-arm 22. A 
tracking shield and grid 34 , such as fiducial grid 34 , are fitted onto the image intensifier 
of the C-ann 22. The grid 34 contains a set of markers 36 that are visible in images 24, 
and allow the image 24 projection to be determined accurately. The position sensors 36 
with the tracked fiducial grid 32 are used to calibrate and/or register medical images 24 
by fixing the position of the grid relative to the patient trackers 30 at the time the image 
24 is acquired. 

[00301 The system 10 also includes hardware and electronics used to synchronize 

the moment of images 24 acquisition to the tracked position of the patient 12 and/or 
imaging device 22 . The systems 10 also includes electronics to communicate signals 
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from the position sensors 30, 36,38 or communicate measurements or information to the 
computing device 26 or electronics to the computing device 26 or other part of the system 
10. 

[0031] The instruments 16 also include positional sensors 38, or instrument 

trackers that provide an unambiguous position and orientation of the instruments. This 
allows the movement of the instruments 16 to be tracked virtually represented on the 
images 24 in the application program while performing the procedure. Some 
instruments 16 are designed specifically for the navigation system 10, while existing 
orthopedic instruments 16 can be adapted to work with the navigation system 10 by 
rigidly attaching trackers 34 to some part of the instrument 16 so that they become visible 
to the camera. By virtue of a tracker attached to an instrument, the position and trajectory 
of the instrument in the 3D coordinate system, and therefore relative to the patient can be 
determined. The trackers 38 fit onto the instruments 16 in a reproducible location so that 
their relation can be pre-calibrated. Verification feat this attachment has not changed is 
provided with a verification device. Such a verification device contains "docking 
stations" where the instruments 16 can be positioned repeatedly relative to fixed locations 
and orientations. Existing instruments can be adapted by securing abutments on to the 
surgical instruments in a known position/orientation with respect to the instrument's 
axes. The calibration can be done by registering the position when in the docking station 
with a calibration device and storing and associating this calibration information with the 
particular docking station. 

{0032] Alternatively, the docking station could be mechanically designed such 

that it has a unique position for the instrument in the docking station and such that the 
calibration information could be determined through the known details and configuration 
of the instrument 

[0033] Accordingly, the instrument and its associated tracker, can be removed 

from the docking station and its position monitored. 

(0034] Similarly, the implants 20 include trackers 36 which may be integrated in 

to the implant or detachably secured so as to be disposable after insertion. The trackers 
36 provide positional information of the implant 20 detectable by the system 10. The 
devices 36 transmit a signal to the tracking system 27 regarding their identity and 

7 



BNSDOCID: <W O 2004O0 1 569A2 J _> 



SUBSTITUTE SHEET (RULE 26) 



PCT/CA2003/000947 

WO 2004/001569 



position. The trackers on the devices 36 may include embedded electronics for 
measurement, computing and display allowing them to calculate and display values to the 
system 10 or directly to the user and may include a user-activated switch. 
(0035] Images 24 of the patient 12 are taken and landmarks identified after 

patient trackers are rigidly mounted and before surgical patient positioning and draping 
on a surgical table 14. The images 24 are manually or automatically "registered" or 
"calibrated" by identification of the landmarks on bom the patient and image 24 . Since 
the images 24 are registered and saved on me computer readable medium of the 
computing device with respect to the tracker location, no more imaging may be required, 
unless required during the procedure. Therefore there is minimal radiation exposure to 
the user 18. 

(0036] To assist in the planning of the procedure, the computing device of the 

system 10 includes stored images 24 of implants and instruments compatible to the 
imaging system utilised. With an X-ray device, the images 24 are generated by an 
algorithm for generating a 2D projection of instruments 16 and implants 22 onto 2D X- 
ray images 24. This involves algorithms that take the 3D CAD information and generate 
a 2D template that resembles templates that surgeons 18 are familiar with for planning. 
For example, the projection of the 3D femoral stem and acetabular cup model onto the X- 
ray is performed using a contour -projection method mat produces the dynamic template 
that has some characteristics similar to the standard 2D templates used by surgeons 28, 
and therefore is more intuitive. 

[00371 The "dynamic 2D template" from the 3D model provides both the exact 

magnification and orientation of the planned implant on the acquired image 24 to 
provide an mtuitive visual interface. A 2D template generation algorithm uses the 3D 
geometry of the implant, and 3D-2D processing to generate a projection of the template 
onto the calibrated X-ray image 24 . The 2D template has some characteristics similar to 
those provided by implant manufecturers to orthopaedic surgeons for planning on planar 
X-ray films. The application program 32 allows the user to maneuver the virtual images 
24 of prosthetic components or implants until the optimum position is obtained. The 
surgeon can dynamically change the size of component among those available until the 
optimum configuration is obtained. 

8 
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[0098] To facilitate the actual procedure, the system 10 also automatically detects 

implant and/or instrument models, by reading the bar codes carried by the implants. The 
system 10 includes a bar code reader that automatically or semi-automatically recognizes 
a cooled opto-reflecting bar code on an implant 20 package by bringing it in the vicinity 
of a bar code reader of the system 10 • The implants are loaded into the system 10 and 
potentially automatically registered as a "used inventory" item. This information is used 
for the purposes of inventory control within a software package that could be connected 
to the supplier's inventory control system that could use this information to remotely 
track supplier and also replenished when a system 10 indicates that it has been used. 
Each of the implants carries trackers that are used to determine the orientation and 
position relative to the patient and display that on the display 28 as an overlay of the 
patient image 24 . 

[0039] It is recognized that other active/passive tracking systems could be used, if 

desired. The tracking system 27 can be, but is not limited to optical, magnetic, 
ultrasound, etc. Could also include hardware, electronics or internet connections that are , 
used for purposes, such as remote diagnostics, training, service, maintenance and 
software upgrades. Other tracking means electrically energizeable emitters, reflective 
markers, magnetic sensors or other locating means. 

[0040] Each surgical procedure includes a series of steps such that there is a 

workflow associated with each procedure. Typically, these steps or tasks are completed in 
sequence. For each procedure the workflow is recorded by a workflow engine 38 in 
coupled to the application program 32. Thus, the system 10 can guide the user 18 by 
prompting the user 18 to perform the task of the workflow or the user 18 directs the 
workflow to be followed by the system 10 by recognizing the tracked instruments 16 as 
chosen by the user 18. Generally, a combination of both guided workflows are possible 
in any given procedure. Thus, the user 18 can trigger an action for a specific workflow 
task. When the system 10 detects that a given task of the procedure has been invoked, it 
displays the required information for that procedure, pertinent measurements, and/or 
medical images 24. The system 10 also automatically completes user 18 input fields to 
specify certain information or actions. The guide also alerts the user 18 if a step of the 
workflow has been by-passed. 
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[0041] The tasks of the procedure are invoked by the user 18 interacting with the 

system 10 via an interface sub-system 40. The user 18 includes position sensors 42 or 
user trackers, typically mounted on the user's 18 hand. These sensors 42 provide tracking 
of user's 18 position and orientation. Generally, a hand input device 44 with attached 
tracker 42 or an electroresistive sensing glove is used to report the flexion and abduction 
of each of the fingers, along with wrist motion. Thus, each task of the workflow is 
associated with hand gestures, the paradigm being gesturally-based hand gestures to 
indicate the desired operation. 

10042] Hand gestures may also be used during planning. For example, the user 

18 could make the "drill" gesture and the corresponding image 24 , i.e. a virtual drill is 
called from the instrument image database and applied to the patient 12 data (hip) in the 
environment Similarly, a sawing motion invokes the femoral proximal cut guidance 
mode, while a twisting motion invokes a reamer guidance mode and shows a rasp to 
invoke the leg length and anteversion guidance mode. Hand gestures may also be used 
during the surgical procedure to invoke iteration of the work flow steps or other action 
required. 

[0043] Prior to the start of the procedure, a plurality of hand gestures are 

performed by the user 18, recorded by the computing device 22, and associated with a 
desired action and coupled to the pertinent images 24, measurement data and any other 
information specific to that workflow step. Therefore, if during the procedure, the user 18 
performs any of the recorded gestures to invoke the desired actions of the workflow; the 
camera detects the hand motion gesture via the position sensors 42 and sends this 
information to the workflow engine for the appropriate action. Similarly, the system 10 
is responsive to the signal provided by the individual instruments 16, and, responds to the 
appearance of the instruments in the field of vision to initiate actions in the work flow. 
The gestures may include a period of time in which an instrument is held stationary or 
may be combinations of gestures to invoke certain actions. 

[0044] The steps for a typical method of a computer assisted surgery system 10 

will now be described with the aid of a flowchart in Figure 5. 

[0045] Initially, patient trackers 30 are attached onto the patient 12 by suitably 

qualified medical personnel 18, and not necessarily by a surgeon 18 . This attachment of 
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trackers may be done while the patient 12 is under general anesthesia using local 
sterilization. The patient image 24 is obtained using the C-arm 22 or similar imaging 
technique, so that either registration occurs automatically or characteristic markers or 
fiduciaries may be observed in the image 24 . The markers may be readily recognized 
attributes of the anatomy being imaged, or may be opaque <c buttons" that are placed on 
the patient. 

[0046] The next step 102 involves calibrating the positional sensors or trackers on 

the instruments 16, implants 20 and a user's 1 8 hand in order to determine their position 
in a 3- dimensional space and their position in relation to each other. This is 
accomplished by insertion of the verification block that gives absolute position and 
orientation. 

[0047] In the next step 1 04, a plurality of hand gestures are performed by the user 

18 and recorded by the computing device 22. These hand gestures are associated with a r 
desired action of the workflow protocol; 

[0048] Registration is then performed if necessary between the image and patient 

by touching each fiduciary on the patient and image in succession. In this way, the image 
is registered in the 3D framework established by the cameras to that the relative 
movement between the instruments and patient can be displayed. 
[0049] . The next steps involves planning of the procedure. At step 1 10 the 

position of the patient's 12 anatomical region is registered. This step includes the sub- 
steps of tracking that patient's 12 anatomical region in space and numerically mapping it 
to a corresponding medical images 24 of that anatomy. This step is performed by 
locating some anatomical landmarks on the patient's 12 anatomical region with the 3D 
tracking system 27 and in the corresponding medical images 24 and calculating the 
transformation between 3D tracking and medical images 24 coordinate systems. 
[0050] At step 1 12, the 2D templates of the instruments and implants generate a 

projection of the template onto the calibrated 2D X-ray images 24 in real time* The 
"dynamic 2D template" from the 3D model provides both the exact magnification and 
orientation of the planned implant with the intuitive visual interface. This step also 
includes generating a 2D projection of instruments 16 onto 2D X-ray images 24. The 
instruments 16 to be used on the patient 12 while performing the procedure are virtually 
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represented on the images 24, and so are the implants. The 3D implant and instrument 
geometric models in combination are used with the registered medical images 24, and the 
generating 2D projections of that instrument and/or implant are updated dynamically in 
real-time as the implant/instrument is moved about in 3D space. Advantageously, the 
dynamic 2D projection is more intuitive and provides ease of use for a user 1 8 . As the 
steps of the procedure are simulated, datums or references may be recorded on the image 
24 to assist in the subsequent procedure. 

(0051 ] In the next 1 14, a path for the navigation of the procedure is set and the 

pertinent images 24 of the patient's 12 anatomical region are complied for presentation to . 
the user 18 on a display. Thus the user 18 is presented with a series of workflow steps to 
be followed in order to perform the procedure. 

[0052] . After the planning stages, the procedure is started at step 1 1 6 by detecting 
a desired action from the user's hand gestures stored on said computer-readable medium; 
or from the positional information of a tracked instrument with respect to the tracking 
system 27 or other tracked device, or a combination of these two triggers; 
(0053] The next step 118 involves performing the desired action in accordance 

with the pre-set path. However, the user 18 may deviate from the pre-set path or 
workflow steps in which case the system 10 alerts the user 18 of such an action. The 
system 10 provides visual, auditory or other sensory feedback to indicate when that the 
surgeon 18 is off the planned path. The 2D images 24 are updated, along with virtual 
representation of the implant 20 and instrument 1 6 positioning, and relevant 
measurements to suit the new user 1 8 defined path. After each step in the work flow, the 
user 18 increments the task list by gesturing or by selection of a different instrument. 
During the procedure, the references previously recorded provide feedback to the user 18 
to correctly position and orientate the instruments and implants. 
[0054] The method and system 10 for computer assisted surgery will now be 

described with regards to specific examples of hip and knee replacement. Hip 
replacement involves replacement of the hip joint by a prosthesis that contains two main 
components namely an acetabular and femoral component. The system 10 can be used to 
provide information on the optimization of implant component positioning of the 
acetabular component and/or the femoral component. The acetabular and femoral 
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components are typically made of several parts, including for example inlays for friction 
surfaces, and these parts come in different sizes, thicknesses and lengths. The objective of 
this surgery is to help restore normal hip function which involves avoidance of 
impingement and proper leg length restoration and femoral anteversion setting. 
[0055] In a total Hip or MIS Hip replacement guidance method, the clinical 

workflow starts with attachment of MIS ex-fix style patient trackers 30 in figure 5 on the 
patient's 12 back while under general anesthesia using local sterilization. The pins that 
fix the tracker to the underlying bone can be standard external fixation devices available 
on the market onto which a patient tracker is clamped. The user 18 interface of the 
system 10 prompts the user 18 to obtain the images 24 required for that surgery and 
associates the images 24 with the appropriate patient tracker 30. Once the images 24 have 
been acquired, the patient trackers 30 are maintained in a fixed position so that they 
cannot move relative to the corresponding underlying bone. 

[0056] The system 10 presents images 24 that are used to determine a plurality : . 

of measurements, such as the trans-epicondyiar axis of the femur for femoral anteversion 
measurements. Femoral anteversion is defined by the angle between a plane defined by 
the trans-epicondylar axis and the long axis of the femur and the vector of the femoral 
neck To determine the orientation of the transcondylar axis of the femur, the C-arm 22 is 
aligned until the medial and lateral femoral condyles overlap in the sagittal view. This 
view is a known reference position of the femur that happens to pass through the 
transcondylar axis. The orientation of the X-ray image 24 is calculated by the system 10 
and stored in the computer readable medium for later use. The transcondylar axis is one 
piece of the information used to calculate femoral anteversion. 
[0057] The system 10 includes intraoperative planning of the acetabular and 

femoral component positioning to help choose the right implant components, achieve the 
desired anteversion/inclination angle of the cup, anteversion and position of the femoral 
stem for restoration of patient 12 leg length and anteversion and to help avoid of hip 
impingement. Acetabular cup alignment is guided by identifying 3 landmarks on the 
pelvis that defines the pelvic co-ordinate system 1 0 . These landmarks can be the left & 
right cases and pubis symphysis (See Figure 6) 
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[0058] The position of the landmarks can be defined in a number of ways. One 

way is to use a single image 24 to refine the digitized landmark in the ante-posterior (AP) 
plane, as it is easier to obtain an AP image 24 of the hip than a lateral one due to X-ray 
attenuation through soft tissue. This involves moving the landmark within the plane of 
the image 24 without affecting its "depth" with respect to the X-ray direction of that 
image 24, as it is easier to obtain a single AP image 24 of the pelvis due to X-ray 
attenuation of the lateral image 24. The user 18 is made aware that the depth of the 
landmark must have been accurately defined through palpation or bi-planar digitization. 
Use of single X-ray images 24 can be used to ensure that the left and right axes are at the 
same "height" with respect to their respective pelvic crests and to ensure that the pubis 
symphysis landmark is well centered. 

(0059] Alternatively, bi-planar reconstruction from two non-parallel images 24 

of a given landmark can be used. This helps to minimize invasive localization of a 
landmark hidden beneath soft tissue or inaccessible due to patient 12 draping or 
positioning. The difference between modifying a landmark through bi-planar 
reconstruction and modifying the landmark position with the new single X-ray image 24 
technique is that in bi-planar reconstruction, modification influences the landmark's 
position along an "x-ray beam" originating from the other image 24, whereas die single 
X-ray image 24 modification restricts landmark modification to the plane of that image 
24.- 

[0060] The pelvic co-ordinate system 10 is used to calculate an 

anteversion/inclination angle of a cup positioner for desired cup placement. This can also 
be used to calculate and guide an acetabular reamer. The system 10 displays the 
anteversion/inclination angle to the user 18 along with a projection of the 3D cup 
position on X-ray images 24 of the hip. The details of calculations can be seen in figure 
6. 

[0061] For minimal invasive procedures, the system 10 provides navigation of a 

saw that is used to resect the femoral head. This step is performed before the acetabular 
cup guidance to gain access to the acetabulum. The system 10 displays the relevant C- 
arm 22 images 24 required for navigation of the saw and display the saw's position in 
real-time on those images 24. Guidance may be required for determining the height of 
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the femoral cut The system 10 then displays the relevant images 24 for femoral reaming 
and displays the femoral reamer. If the user 18 has selected an implant size at the 
beginning or earlier in the procedure, the system 10 displays the reamer corresponding to 
this implant size. Note that since reaming process starts with smaller reamers and works 
it's way up to the implant size, the virtual representation of the reamer will be larger than 
the actual reamer until the implant size is reached (for example for a size 12 implant, the 
surgeon 1 8 will start with a 8-9mm reamer and work up in l-2mm increments in reamer 
size). This virtual representation allows the surgeon 18 to see if the selected implant size 
fits within the femoral canal. Secondly, it can help avoid the user 1 8 having to change 
the virtual representation on the UI for each reamer change which often occurs very 
quickly during surgery (time saving). The user 18 is able to change the reamer diameter 
manually if required. 

[0062] The system 10 assists in guiding the orientation of the femoral reaming' 

in order to avoid putting the stem in crooked or worse notching the intra-medullary canal, 
which can cause later femoral fracture,. A virtual representation of the reamer and a 
virtual tip extension of the reamer are provided so the surgeon 18 can align the reamer 
visually on the X-ray images 24 to pass through the centre of the femoral canal. The 
system 10 allows the surgeon 18 to set a current reamer path as the target path. The 
system 10 provides a sound warning if subsequent reamers are not within a certain 
tolerance of this axis direction. 

[0063] The femoral anteversion calculation is described below with the aid of 

Figure 6: 

where « pro bc be a unit vector, pointing from the tip of the cup imjpactor towards the 
handle, and 

flftomab «axiai, and Wsaggitat, are unit vectors that are normal to the three orthogonal planes 
that form the pelvic co-ordinate system. 

ifixmtai be a unit vector, normal to the frontal plane of the patient 12 , whose sense is from 
the posterior to the anterior of the patient 12 . 

"axial be a unit vector, normal to the axial plane of the patient 12 , whose sense is from the 
inferior to the superior of the patient 12 . 
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nsaggw be a unit vector, normal to the sagittal plane of the patient 12 , whose sense is 
from patient 12 right to patient 12 left. 
Let a. represent the anteversion. 
Let 0 represent the inclination. 

Vpn.befiom.1 = ("p«>be • "axial) "axial + ("probe ' "sagittal) "sagittal 

"probe_fronta1 — Vprobe_fronta! / |Vprobe_frontal| 
CX= 90 - COS"'(«frontal ' "probe) 

fi= Sign X { 1 80 - COS'^Waxial ' "probefrontaO} 

where: 

For a left hip 'sign* is positive unless iiprobefrontai " "sagittal < 0. 
For a right hip 'sign' is positive unless »p«>be_&ontai ' "sagittal > 0. 

(00641 The system 10 also provides a technique for obtaining the trans-epicondylar 
axis of the femur. An accepted radiological reference of the femur is the X-ray view 
where the distal and posterior femoral condyles overlap. The direction of this view also 
happens to be the trans-epicondylar axis. The fluoro-based system 10 tracks the position 
of the image 24 intensifier to determine the central X-ray beam direction through C-arm 
22 image calibration. The epicondylar axis is obtained by acquiring a C-arm 22 image 
that aligns the femoral condyles in the sagittal plane and recording the relative position of 
the C-arm 22 central X-ray beam with respect to the patient tracker. 
[006SJ Once these vectors are defined in the workflow, the system 10 will 

provide real-time update of femoral anteversion for a femoral rasp and femoral implant 
guides. A femoral rasp is an instrument inserted into the reamed femoral axis and used to 
rasp out the shape of the femoral implant. It is also possible to provide femoral 
anteversion measurements for other devices that may be used for anteversion positioning 
(for example the femoral osteotome). The system 10 also updates in real-time the effect 
ofrasporimplantpositiononleglength. Leg Length is calculated in three steps. In the 
first step, before the hip is dislocated, the distance between a femoral tracker, 2>, and a 
pelvic tracker, 2>.are obtained. Therefore, the initial distance, U =(?>- T p ) • n a 
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[0066] . The second step of the process involves calculating the new leg length 
fraction attributed to the acetabular cup position, L c . Once the cup has been placed, the 
position of the cup impactor, P u is stored After the acetabular cup shell and liner have 
been selected, the exact location of the center of rotation along the impactor axis, P e is 
obtained from the 3D models of the implants. The center of rotation is then projected 
onto the pelvic normal and relative to the pelvic tracker, and the length attributed by cup 
position, L c = P c • n a 

[0067] In the next step, the new leg length fraction attributed to the femoral stem 

position, L Si is obtained. After selection of the desired stem and head implants, the 
precise location of the femoral head is obtained from the 3D models of the implants, /V 
As the femur is being rasped, the length is continuously calculated along the anatomical 
axis of the femur, V/emur, relative to the femoral tracker, T/ by monitoring the position of 
the reamer. The length attributed to stem position, L s = P h * V femur 
[0068] The implant models and components can be changed "on the fly" and the 

resulting effect on the above parameters displayed in real-time by the computer- 
implemented system 10. As indicated in figure 10, the application program implements 
algorithms which take into consideration changes in parameters such as component shape 
size and thickness to recalculate leg length and anteversion angles. Intra-operative 
planning may be important in hips or knees where bone quality is not well known until 
the patient 1 2 is open and changes in prosthesis size and shape may need to be 
performed intra-operatively. When a new component is chosen or when the surgeon 18 
rasps further down into the femur than planned, due to poorer than expected bone quality 
for example, the system 10 will automatically generate updated leg length measurements 
and anteversion angles so that in situ decisions can be made. For example if the surgeon 
18 has rasped too far into the femur, which would result in a leg length loss, the system 
10 could be used to see if a larger sized femoral neck length or larger size femoral 
implant could be used to maintain the correct leg length. 

[0069] The system 10 also calculates potential impingement in real-time between 

femoral and acetabular components based on the recorded acetabular cup position and the 
current femoral stem anteversion. Implant-implant impingement calculation is based on 
the fact that the artificial joint is a well-defined ball and socket joint. Knowing the 
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acetabular component and femoral stem component geometry, one can calculate for 
which clinical angles impingement will occur. If impingement can occur within angles 
that the individual is expected to use, men the surgeon 18 is warned of potential 
impingement. Once the acetabular component has been set, the only remaining degree of 
freedom to avoid impingement is the femoral anteversion. 

[0070] As mentioned above, the system 10 generates a 2D projection of implants 

onto 2D X-ray image 24 to provide the surgeon 18 with a more familiar representation., 
as shown in Figure 1 1 . The 2D projection model would be updated as the implant is 
rotated in 3D space. 

[0071] The system 1 0 can also optionally record information such as the position 

of the femoral component of the implant or bony landmarks and use this information to 
determine acetabular cup alignment mat minimizes the probability of implant 
impingement. This can help guide an exact match between acetabular and femoral 
anteversion for component alignment The system 10 can help guide the femoral reamer 
that prepares a hole down the femoral long axis for femoral component placement to 
avoid what is termed femoral notching that can lead to subsequent femoral fracture. The 
system 10 provides information such as a virtual representation of the femoral reamer on 
one or more calibrated fluoroscopy views, and the surgeon 18 can optionally set a 
desired path on the image 24 or through the tracking system 21, and includes alerts 
indicative of the surgeon 18 straying from the planned path. 
(00721 The system 10 guides the femoral rasp and provides femoral axis 

alignment information such as for the femoral reamer above. The chosen rasp position 
usually defines the anteversion angle of the femoral component (except for certain 
modular devices that allow setting of femoral anteversion independently). Femoral 
anteversion of the implant is calculated by the system 10 using information generated by 
a novel X-ray fiuoroscopy-based technique and tracked rasp or implant position. It is 
known that an X-ray image 24 that superimposes the posterior condyles defines the trans- 
epicondylar axis orientation. If the fiducial calibration grid 34 is at a known orientation 
with respect to the X-ray plane in the tracking system 27 (either through design of the 
fiducial grid 34 or through tracking of both the fiducial grid 34 and the C-arm 22 ), the 
system 10 knows the image 24 orientation and hence the trans-epicondylar axis in the 
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■t tracking co-ordinate system 10 . The system 10 then can provide the surgeon 18 with 
real-tirrie feedback on implant anteversion based on planned or actual implant position 
with respect to this trans-epicondylar axis. Alternative methods of obtaining the trans- 
epicondylar axis include direct digitization or functional rotation of the knee using the 
tracking device. 

I0073J Proper femoral anteversion is typically important to help avoid 

impingement, as is the anteversion/inclination angle of the acetabular component Since 
impingement occurs due to the relative orientation between the acetabular and femoral 
components, the system 10 optimizes femoral anteversion based on the acetabular 
component orientation if the latter was recorded by the tracking 27 as described above. 
(00741 The effect of implant position on leg length, femoral anteversion and 

"impingement zone" is updated in real-time with the planned or actual implant position 
taking into account the chosen acetabular component position. Implant model and < 
components can be changed "on the fly" and used by the surgeon 18 through and the 
resulting effect on the above parameters displayed in real-time. 
(00751 The technology involves "intelligent instruments" that, in combination 

with the computer, "know what they are supposed to do" and guide the surgeon 18 . The 
system 10 also follows the natural workflow of the surgery based on a priori knowledge 
of the surgical steps and automatic or semi-automatic detection of desired workflow K 
steps. For example, the system 10 provides the required images 24 and functionality for 
the surgical step invoked by a gesture. Specific examples of gestures within the hip 
replacement surgery include picking up the cup positioner to provide the surgeon 18 with 
navigation of cup anteversion/inclination to within one degree (based on identification of 
the left & right axes and pubis symphysis landmarks), picking up the reamer and the rasp 
will also provides the appropriate images 24 and functionality, while picking up the saw 
will provide interface for location and establishment of the height that the femoral head 
will be cut. The surgeon 18 can skip certain steps and modify workflow flexibly by 
invoking gestures for a given step. The system 10 manages the inter-relationships 
between the different surgical steps such as storing data obtained at a certain step and 
prompting the user 18 to enter information required for certain. 
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[0076] Disposable components for a hip instrumentation set include a needle 

pointer, a saw tracker, an optional cup reamer tracker, a cup impactor tracker, a drill 
tracker (for femoral, reamer tracking), a rasp handle tracker, a implant tracker, and a 
calibration block. 

[0077] In another example, the system 10 is used for a uni-condylar knee 

replacement. The uni-knee system 10 can be used without any images 24 or with fluoro- 
imaging to identify the leg's mechanical axes. The system 10 allows definition of hip, 
knee and ankle center using palpation, center of rotation calculation or bi-planar 
reconstruction. 

[0078] The leg varus/valgus is displayed in real-time to help choose a uni- 

compartmental correction or spacer. The surgeon 18 increases the spacer until the 
desired correction is achieved. Once the correction is achieved, the cutting jig is put into 
place for the femoral cut The tibial cuts and femoral cuts can be planned n virtually" 
based on the recorded femoral cutting jig position before burring. In the case of a system 
10 that uses a burr to prepare the bone, two new methods for guiding the burr are 
particularly beneficial. The first is a "ftee-hand" guide that tracks the burr. A cutting 
plane or curve is set by digitizing 3 or more points on die bone surface that span the 
region to be burred. The system 10 displays a color map representing the burr depth in 
that region and the color is initially all green. The desired burr depth is also set by the 
user . As the surgeon 1 8 burrs down, the color at that position on the colormap turns 
yellow, orange then red when the burr is within 1mm of desired depth (black will indicate 
that burr has gone too far). The suggested workflow is to <c borrow" burr holes at the 
limits of the area to be burred down to the red zone under computer guidance. The 
surgeon 18 then burrs in between these holes only checking the computer when he/she is 
unsure of the depth. The system 10 can also provide sound or vibration feedback to 
indicate burring depth. 

[0079] A small local display or heads-up display can help the surgeon 1 8 

concentrate on the local situs while burring. For the curved surface of the femur, the 
colormap represents the burr depth along a curve. 

{00801 The second method presented is a passive burr-guide. The following is an 

example: a cutting jig has one to four base pins and holds a "burr-depth guide" that 
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restricts burr depth to the curved (in femur) or flat (in tibia) implant The position and 
orientation of this device is computer guided (for example by controlling height of bun- 
guide on four posts that place it onto the bone). The burr is run along this burr guide to 
resect the required bone. As in the hip replacement procedure, the patient trackers 30 are 
positioned similarly. 

[0081] The system 10 can also be linked to a pre-operative planning system in a 

novel manner. Pre-operative planning can be performed on 2D images 24 (from an X- 
ray) or in a 3D dataset (from a CT scan). These images 24 are first corrected for 
magnification and distortion if necessary. The implant templates or models are used to 
plan the surgery with respect to manually or automatically identified anatomical 
landmarks. The pre-operative plan can be registered to the intra-operative system 10 
through a registration scheme such as corresponding landmarks in the pre and intra- 
operative images 24. Other surface and contour-based methods are also alternative 
registration methods. In the case of hip replacement, for example, the center of the 
femoral head and the femoral neck axis provide such landmarks that can be used for 
registration. Once these landmarks have been identified intra-operatively, the system 10 
can position the planned implant position automatically, which saves time in surgery. 
The plan can be refined intra-operatively based on the particular situation, for example if 
bone quality is not as good as anticipated and a larger implant is required. 
[0082] Although the invention has been described with reference to certain 

specific embodiments, various modifications thereof will be apparent to those skilled in 
the art without departing from the spirit and scope of the invention as outlined in the 
claims appended hereto. 
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THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE 
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS: 

1. A computer-implemented method for enhancing interaction between a user and a 
surgical computer assisted system, the method includes the steps of: 

tracking a user's hand gestures with respect to a reference point; 

registering a plurality of gesturally-basedhand gestures and storing said gestures 

on a computer-readable medium; 

associating each of said plurality of gesturally-based hand gestures with a desired 
action; 

detecting a desired action by referencing said user's hand gestures stored on said 
computer-readable medium; and 
performing the desired action. 

2. A method of claim 1 wherein said hand gestures that is tracked by a tracking, 
system through an instrument in the user's hand. 

3. A method of claim 2 where said instrument includes a tracking module for 
providing identification of said instrument, said identification being recognizable 
by the tracking system. 

4. A method of claim 3 wherein said instrument is associated with a desired action . 

5. A method of claim 4 wherein said desired action is detected by analyzing timing 
information related to an amount of time the instrument is held in a certain 
position with respect to reference point 

6. The method of claim 1 wherein said a desired action can detected from 
combination of hand gestures. 

7. A method of claim 1 wherein the hand gestures are associated with desired 
actions in a workflow of the user. 

8. A method of claim 7 wherein said workflow includes a set of desired steps and 
when the system detects that a given step of the procedure has been invoked, it 
configures a user interface to provide the required information, such as 
measurements and/or medical images, and provides required functionality, such 
as user input fields to specify certain information or actions. 
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9. The method of claim 8 wherein the user can access any step of the procedure in 
any given order, and the system prompts the user to pass to a subsequent step if 
there is missing information. 

10. A method of claim 1 where the system automatically detects an implant and/or 
instrument model. 

1 1. A method of claim 10 wherein the said implant is encoded with the identifying 
information. 

12. A method of claim 1 1 wherein said implant has a bar code readable by a bar code 
reader. 

13. A method according to claim 12 wherein said implant package has a coded opto- 
reflecting bar-code information recognizable by the tracking system. 

14. A method of claim 1 1 wherein said implant having been detected by the system is 
automatically registered as a "used inventory" item. 

15. A method of a computer assisted surgery system to reduce user interaction, by 
determining information for a surgical procedure from the orientation of a 
medical image from an imaging device. 

16. A method of claim 15 wherein information regarding the orientation of the 
medical image is obtained by tracking said imaging device or tracking of a 
fiducial object associated with said imaging device and visible in the image. 

17. A method of claim 16 wherein the orientation of the imaging device is used to 
determine the medio-lateral axis of a femur, said axis being used for 
biomechanical calculations of a leg. 

18. A method of a computer assisted surgery system including the steps of attaching 
minimal invasive patient trackers to a patient, acquiring intraoperative images 
with respect to said patient trackers, registering said intraoperative images to 
those trackers and storing said intraoperative images on a computer readable 
medium, whereby said intraoperative images are used during surgery precluding 
the use of said imaging device during surgery. 

19. A method of a computer assisted surgery system having the step of displaying a 
magnified virtual representation of a target instrument or implant size while 
smaller instruments or implants are being used. 
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20. A method of assisting a surgical procedure by obtaining an image of a portion of a 
patient on whom the procedure is to be performed, 

registering said image and said patient in a three dimensional coordinate 
system, 

monitoring movement of said patient in said three dimensional coordinate 
system, 

monitoring movement of an instrument to be used in said procedure, 
superimposing a representation of said instrument of said image and 

adjusting relative positions thereof on said image as the relative position of said 

instrument and said patient vary, 

monitoring movement of an implant to be used in said procedure and 

supervising a representation thereof on said image and adjusting the relative 

position thereof on said image as the relative position of said implant and patient 

varies. 

21. A method according to claim 20 wherein reference data is provided oh said image 
to assist in positioning said instrument and/or implant 
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Let /iproiU be a unit vector, pointing from the tip of the cup impactor towards the 
handle. 

Let ifftoatai, »axiaL and /tsaggitaL be unit vectors that are normal to the three orthogonal 
planes that form the pelvic co-ordinate system. 

Let iffiorfal be a unit vector, normal to the frontal plane of the patient, whose sense 
is from the posterior to the anterior of the patient. 

Let ffaxiai be a unit vector, normal to the axial plane of the patient, whose sense is 
from the inferior to the superior of the patient. 

Let "jaggitai be a unit vector, normal to the sagittal plane of the patient, whose sense 
is from patient right to patient left. 
Let a represent the ante version. 
Let p represent the inclination 

v piob*_fioiTtal — ("probe " "axial) "axial + ("puobe * "sagittal) "sagittal 

"piobe_fiantal = **p robe _fto ntal fiantall 
a= 90 — COS" l («fiontal " "paobe) 



P = Sign X {180 - COS'^Waxial ' «piobe_fiurtal)} 



where: 

For a left hip 'sign is positive unless /* pro be_fianial • "sagittal < 0. 
For a right hip 'sign is positive unless /* P iobe_fiartai " "sagittal > 0. 



Figure 7 



BNSOOCID: <W O 20Q4QQ 1 569A2 J _> 



SUBSTITUTE SHEET (RULE 26) 




BNSOOCID: <WO__2004001569A2.I_> 



SUBSTITUTE SHEET (RULE 26) 



WO 2004/001569 PCT/CA2003/000947 

9/11 




Figure 9 



BNSOOC1D: <WO__2004001569A2_I_> 



SUBSTITUTE SHEET (RULE 26) 



PCT/CA2003/000947 

WO 2004/001569 

10/11 




BNSDOCJD: <WO 2 0O4OQ1 S69A2J_> 



SUBSTITUTE SHEET (RULE 26) 



WO 2004/001569 



11/11 



PCT/CA2003/000947 




Figure 1 1 



SUBSTITUTE SHEET (RULE 26) 

BNSOOCID: <WO 200400 1 569A2J _> 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property 
Organization 

International Bureau 




laiiioiiiiiiiiiiiiiiiiiniiDiu 



(43) International Publication Date (™) International Publication Number 

31 December 2003 (31.12.2003) PCT WO 2004/001569 A3 



(51) International Patent Classification 7 : G06F 3/00, 

A61B 19/00 

(21) International Application Number: 

PCT/CA20O3/O0O947 

(22) International Filing Date: 23 June 2003 (23.06.2003) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) Priority Data: 
60/390,188 



2 1 June 2002 (2 1 .06.2002) US 



(71) Applicant (for all designated States except US): CEDA RA 
SOFTWARE CORP. [CA/CA]; 6509 Airport Road, Mis- 
sissauga, Ontario L4V 1S7 (CA). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): SATI, Marwan 
[CA/CA]; 3355 Spirea Terrace, Mississauga, Ontario L5N 



7N6 (CA). CROITORU, Haniel [CA/CA]; 420 Ellerslie 
Avenue, Toronto, Ontario M2R iC2 (CA). TATE, Peter 
[CA/CA]; 450 Sl David Street North, Fergus, Ontario 
N1M 2K2 (CA). FU, Hqun (CA/CA); 7369 Glamorgan 
Way, Mississauga, Ontario L5N 7Z3 (CA). 

(74) Agent: ORANGE, John, R. S.; McCarthy Tetrault LLP, 
66 Wellington Street West, Suite 4700, Box 48, Toronto 
Dominion Bank Tower, Toronto, Ontario M5K 1E6 (CA). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EC, EE, ES, FI, GB, GD, GE, GH, 
GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, ICR, KZ, LC. 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NO, NZ, OM, PH, PL, PT, RO, RU. SC, SD, SE, 
SG, SK, SL, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, 
VC, VN, YU, ZA, ZM, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 
European patent (AT, BE, BG, CH, CY, CZ, DE, DK, EE, 
ES, FI, FR, GB, GR, HU, IE, IT, LU, MC, NL, PT, RO, 

[ Continued on next page] 



(54) Title: COMPUTER ASSISTED SYSTEM AND METHOD FOR MINIMAL INVASIVE HIP, UNI KNEE AND TOTAL KNEE 
REPLACEMENT 



< 
ON 




O 



(57) Abstract: As a general overview, the system ( 10) is used to assist the surgeon in performing an operation by acquiring and dis- 
playing an image of the patent. Subsequent movement of the patient and instruments is tracked and displayed on the image. Images 
of a selection of implants are stored by the system and may be called to be superimposed on the image. The surgical procedures 
may be planned using the images of the patient and instruments and implants and stored as a series of sequential tasks referred to 
defined datums, such as inclination or position. Gestures of the surgeon may be used in the planning stage to call the image of the 
instruments and in the procedure to increment the planned tasks. 



8NSOOCID: < WO 200400 1 569A3 J _> 



WO 2004/001569 A3 ■■■gggggl 

SE, SI, SK, TR), OAPI patent (BP, BJ, CF, CG, CI, CM, (88) Date of publication of the international search report: 
GA, ON. GQ. GW, ML, MR, NB, SN, TD, TG). June ^ 



For two-letter codes and other abbreviations, refer to the "Guid- 
PubUshed- ance Notes °" Codes am ' Abbreviations' appearing at the begin- 

- with international search report "ing of each regular issue of the PCI Gazette. 



BNSDOCID: <WO 200400 1 S69A3 J _> 



INTERNATIONAL SEARCH REPORT 



Interna^MpI Application No 

PCtJR 03/00947 



A. CLASSIFICATION OF SUBJECT MATTER ^ # ^ 

IPC 7 GO6F3/00 A61B19/00 



According to International Patent Classification (IPC) or to both national dasggcajton and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched {classification system followed by classification symbols) 

IPC 7 G06F A61B 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 

EP0- Internal 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category ° Citation of document, with Indication, where appropriate, of the relevant passages 



Relevant to claim No. 



0E 198 45 028 A (SIEMENS AG) 

8 June 2000 (2000-06-08) 

column 5, line 61 - column 6, line 45 



US 5 880 976 A (BLACKWELL MICHAEL K 
AL) 9 March 1999 (1999-03-09) 
column 9, line 26 - line 44 



ET 



DE 199 60 020 A (MARMULLA RUEDIGER) 
21 June 2001 (2001-06-21) 
column 2, line 32 - line 41 

US 6 201 984 Bl (FUNDA JANEZ ET AL) 
13 March 2001 (2001-03-13) 
-cq.liJOTin : 6, line 23 .- line 59 

DE 39 17 876 A (AESCULAP WERKE AG) 
6 December 1990 (1990-12-06) 
column 2, line 33 - line 64 



1,6-9 

10- 13 
10 

11- 13 



14 



□ 



Further documents are fisted In the continuation of box C. 



Patent family members are Gsted In annex. 



° Special categories of cited documents : 

"A" document defining the general state of the art which is not 

considered to be of particular relevance 
"E" earfier document but published on or after the international 

filing date 

*T_" document which may frirow doubts on priority claim (s) or 
which Is cited to establish the publication date of another 
citation or other special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or 
other means 

"P* document published prior to the international firing date but 
later than the priority date claimed 



T later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

"X" document of particular relevance; the claimed Invention 
cannot be considered novel or cannot be considered to 
involve ah inventive step when the document is taken alone 

• V document of particular relevance; the claimed . invention 

cannot be considered to involve an inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a- person skilled 
In the art 

"&" document member of the same patent family 



Date of the actual completion of the international search 



1 December 2003 



Date of mailing of the International search report 



08. 03. im 



Name and maffing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
NL-2280HVRIjswiJk 
Tel (+31-70) 340-2040, Tx. 31 651 epo nl. 
Fax {+31-70) 340-3016 



Authorized officer 



Angel i , M 



Form PCT/1SA/210 (second shed) (July 1892) 
BNSOOCID: <WO__200400156SA3J_> 



INTERNATIONAL SEARCH REPORT 


Ini^fetiona) application No. 

WPCT/CA 03/00947 




Box 1 Observations where certain claims were found unsearchable (Continuation or item i 01 nrsi sneetj 




•mis International Search Report has not been established in respect of certain claims under Article 1 7(2)(a) for the following reasons: 
1 ' ^ be^e toy relate to subfJt matter not required to be searched by this Authority, namely: 

Rule 39.1(iv) PCT - Method for treatment of the human or animal body by 
surgery 

m 2-5 15-17 

2 - L±J gSELaltay relate to parts of the Internationa! Application that do notcomply with the prescribed requirements to such 

an extent thai no meaningful International Search can be carried out. specifically: 

see FURTHER INFORMATION sheet PCT/ISA/210 

3 - □ b^e*ey are dependent claims and are not drafted in accordance with the second and third sentences of Rule 6.4(a). 




Box II Observations where unity of invention is lacking (Continuation of Hem 2 of first sheet) 




This International Searching Authority found multiple inventions In this International application, as follows: 

see additional sheet 

1. n As all required additional search fees were timely paid by the applicant, this International Search Report covers all 
1 ' searchable claims. 

2 n As an searchable claims could be searched without effort justifying an additional fee. this Authority did not invite payment 
1 — 1 of any additional fee. 

3 n As only some of the required additional search fees were timely paid by the applicant, this International Search Report 
1 1 ewers orty those claims for which fees were paid, specifically claims Nos.: 

4 rn No required additional search fees were timely paid by the applicant Conseqoerrtiy, this International Search Report is 
^ reacted to the invention first mentioned in the claims; it is covered by claims Nos.: 

1, 6-14 . 

Remark on Protest \ | TThe additional search fees were accompanied by the applicant's protest 

| | No protest accompanied the payment of additional search fees. 

l_ ■ ■ : : 



BNSDOC1D: <WO___20O4001 569A3 J_> 



InternationadAppncatlon-No. PCT/CA 03/00947 



FURTHER INFORMATION CONnNUED FROM PCT/ISA/ 210 

This International Searching Authority found multiple (groups of) 
inventions in this international application, as follows: 



1. claims: 1,6-14 

A computer implemented method for enhancing interaction 
between a user and a surgical computer assisted system using 
hand gestures and bar code identification means. 



2. claim: 19 

A method for a computer assisted surgery system displaying a 
magnified virtual representation. 



3. claims: 20,21 

A method of assisting a surgical procedure by monitoring an 
implant and supervising a representation thereof on a 
beforehand obtained image of a patient. 



BNSDOCIO. <WO____20O4O01 569A3_L> 



International Application No. PCT/CA G3/G0947 



FURTHER INFORMATION CONTINUED FROM PCT/ISA/ 210 



Continuation of Box 1.1 
Claims Nos.: 15 

Rule 39.1(iv) PCT - Method for treatment of the human or animal body by 
surgery 



Continuation of Box 1.2 
Claims Nos.: 2-5, 15-17 

extracted from the orientation (Article < PCT). 

examination (Rule 66.He} ^■J; _™i Preli minarv Examining Authority is 
overcome. 



BNSDOCID: <WO___2O04O01569A3J_> 



^J^rmatton on patent family members 


International Application No 

PC^Ia 03/68947 


"Patent document 
cited in search report 


Publication 
date 


Patent family 
members) 


Publication, 
date 





A 

A 




L/C 


IQfc/lcooo A1 
J.7o4DU^O Ml 




IIC £&&A07£ 


A 

A 


fiQLfi? 1 QQQ 


IK 


OC\fO*flL Di. 










IIC 
UJ 


CQOC700 A 


oU-11-1999 








us 


6002859 A. 


14-12-1999 


ut 19950020 


A 


21-06-2001 


□h 


19960020 Al 


01 A^ A/"\ A ^ 

21-06-2001 








AM 
AU 


onno7ni a 
2009701 A 


OP A^ aa/m 

25-06-2001 








AC 

Ut 


200Z2586 Ul 


OA 1 O AAA 1 

20-12-2001 








WO 


0143654 Al 


01 A/" AAA1 

21-06-2001 








EP 


1237493 Al 


11-09-2002 








rn 

EP 


1371340 Al 


1 *Tf « r> aaaa 

17-12-2003 








US 


2002183608 Al 


05-12-2002 


US 6201984 


Bl 


4 AAA1 

13-03-2001 


air* 

US 


5417210 A 


23-05-1995 








US 


5402801 A 


04-04-1995 








US 


5976156 A 


02-11-1999 


• 






AT 


173596 T 


15-12-1998 








DE 


69322202 Dl 


07-01-1999 








DE 


69322202 T2 


01-07-1999 








EP 


0571827 Al 


01-12-1993 








ES v 


2123586 T3 


16-01-1999 








JP 


2575586 B2 


29-01-1997 








JP 


6030896 A 


08-02-1994 








US 


5572999 A 


12-11-1996 










C7lni£0 A 

b/493oZ A 


1 O AC 1 AAA 

12-05-1998 








US 


5445166 A 


29-08-1995 








us 


5695500 A 


09-12-1997 








us 


5950629 A 


14-09-1999 








us 


6024695 A 


15-02-2000 








us 


5630431 A . 


20-05-1997 








us 


6231526 Bl 


15-05-2001 








us 


6547782 Bl 


15-04-2003 








us 


5279309 A 


.18-01-1994 



DE 3917876 A 06-12-1990 DE 3917876 Al 06-12-1990 



Form PCT/ISA/210 (patent tamBy annex) (July 1992) 



BNSOOCJO. <WO__2004001569A3J_> 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property 
Organization 
International Bureau 

(43) International Publication Date 
31 December 2003 (31-12.2003) 




(10) International Publication Number 

PCT WO 2004/001569 A3 



(51) International Patent Classification 7 : G06F 3/00, 

A61B 19/00 

(21) International Application Number: 

PCT/CA2003/000947 

(22) International Filing Date: 23 June 2003 (23.06.2003) 

(25) Filing Language: English 

(26) Publication Language: English 

(30) Priority Data: 

60/390,188 21 June 2002 (21.06.2002) US 

(71) Applicant (for all designated States except US): CEDARA 
SOFTWARE CORR [CA/CA]; 6509 Airport Road, Mis- 
sissauga, Ontario L4V 1S7 (CA). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): SATI, Marwan 
[CA/CA]; 3355 Spirea Terrace, Mississauga, Ontario L5N 
7N6 (CA). CROITORU, Haniel [CA/CA]; 420 Ellerslie 
Avenue, Toronto, Ontario M2R 1C2 (CA). TATE, Peter 
[CA/CA]; 450 St. David Street North, Fergus, Ontario 
N1M 2K2 (CA). FU, Liqun [CA/CA]; 7369 Glamorgan 
Way, Mississauga, Ontario L5N 7Z3 (CA). 



(74) Agent: ORANGE, John, R. S.; McCarthy Tetrault LLP, 
66 Wellington Street West, Suite 4700, Box 48, Toronto 
Dominion Bank Tower, Toronto, Ontario M5K 1E6 (CA). 

(81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EC, EE, ES, FI, GB, GD, GE, GH, 
GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NO, NZ, OM, PH, PL, FT, RO, RU, SC, SD, SE, 
SG, SK, SL, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, 
VC, VN, YU, ZA, ZM, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 
European patent (AT, BE, BG, CH, CY, CZ, DE, DK, EE, 
ES, FI, FR, GB, GR, HU, BB, IT, LU, MC, NL, PT, RO, 
SE, SI, SK, TR), OAPI patent (BF, BJ, CF, CG, CI, CM, 
GA, GN, GQ, GW, ML, MR, NE, SN, TD, TG). 

Published: 

— with international search report 

— with amended claims 

[ Continued on next page] 



(54) Title: COMPUTER ASSISTED SYSTEM AND METHOD FOR MINIMAL INVASIVE HIP, UNI KNEE AND TOTAL KNEE 
REPLACEMENT 



< 

ON 




o 



(57) Abstract: As a genera] overview, the system ( 10) is used to assist the surgeon in performing an operation by acquiring and dis- 
playing an image of the patent. Subsequent movement of the patient and instruments is tracked and displayed on the image. Images 
of a selection of implants are stored by the system and may be called to be superimposed on the image. The surgical procedures 
may be planned using the images of the patient and instruments and implants and stored as a series of sequential tasks referred to 
defined datums, such as inclination or position. Gestures of the surgeon may be used in the planning stage to" call the image of the 
instruments and in the procedure to increment the planned tasks. 



BNSOOCID: <W O 2 0O4OO1 569A3_IA> 



WO 2004/001569 A3 I Hil liyill U Bllll Oil Hid IIU1 1111 1 II 111 IIIII 1IEII Ulll lilll IIUI illl Ulllfl flU IB! mi 



(88) Date of publication of the international search report: 

3 June 2004 

Date of publication of the amended claims: 1 5 July 2004 



For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations " appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



BNSDOC1D: <W O 2 004001 569A3 IA> 



WO 2004/001569 



AMENDED CLAIMS 



PCT/CA2003/000947 



[Received by the International Bureau on 10 May 2004 (10.05.04 ): 
original claims 1-21 replaced by amended claims 1-47] 

THE EMBODIMENT OF THE INVENTION IN WBJCB AN EXCLUSIVE PROPERTY OR 
PRIVELEGE IS CLAIMED ARE DEFINED AS THE FOLLOWING; 

1. A computer-implemented method for enhancing interaction between a user and a surgical 
computer assisted system, the method includes the steps of: 

recognising a placement of a tracked object with respect to a reference point, 

comparing the recognised object placement with a plurality of predefined object placements stored 
in a memory of the computer system, each of the predefined object placements associated with a 
desired action; 

matching the recognised object pLacementvrith the corresponding desired action by referencing the 
stored predefined object placements; and 

coordinating with die user the performance of die desired action. 

2. The method of claim 1, wherein the tracked object is selected from the group comprising a 
surgical instrument and a hand of the user. 

3. The method of claim 2, wherein die placement information of the instrument is selected from the 
group comprising: position; orientation and movement as recognised by a tracking system. 

4. The method of chum 2, wherein the placement of the user's hand includes orientation of the 
fingers ofthe hand as recognised by a tracking system, 

5. The method of claim 4, wherein the predefined object placements are hand gestures. 

6. The method of claim 5 further cotaprising (be step of recognising the hand gestures of the hand 
that is tracked by the tracking system through the instrument located in fee user's handL 

7. The method of claim 2 further comprising the step of providing identification of the instrument by 
a tracking system, said identification being recognizable by the tracking system. 

8. The method of claim 7 further comprising the step of analyzing timing information related to an 
amount of time die instrument is held in a certain position with respect to the reference point, the 
amount of time associated the lack of movement is associated withflie corresponding desired action. 

9. The method of claim 5, wherein the associated desired action is associated with a hand gesture of 
the predefined object placements, 

10. Hie method of claim 5 wherein the hand gestures are associated with the desired actions in a 
workflow of the user. 

1 1. The method of claim 2 further comprising (he step of detecting from a set of desired steps of a 
worMlow, based on the recognised object placement, that a given step of the workflow has been 
invoked, 

12. The method of claim 1 1 further comprising the step of configuring a user interface to provide 
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information related to the given step for interpretation by the user. 

13 The method of claim 11 wherein the user can access a selected step of the workflow in a given 
fa user interface prompts the user to pass to a subsequent step rf there . mtssmg 
infoTtnation. 

14. The method of claim 2 forfcer comjrismg the step of recognising foe object selected from the 
group comprising an implant and foe surgical instrument 

15. The method of claim 14, wherein the implant is encoded with identifying information. 

1 6. The method of claim 15 , wherein the implant has a bar code readable by a bar code reader 
coupled to the compute: system. 

17. The method according to claim 16, wherein the implant has a coded opto-reflecting bar-code 
information recognizable by a tracking system. 

18. The method of claim 15 further comprising&e step of registering said implant oncehaving been 
detected by the computer system as a "used inventory" item. 

19. The method of claim % wherein the desired action is related to a specific workflow task 

20 A surgical corriputer assisted system for enhancing interaction between a user and selected steps 
of a workflow, the system comprising: 

a tacking system fox recognising a placement of a tracked object with respect to a reference point; 

a processor for comparing the recognised object placement with a plurality of predefined object 
placements stored in a memory, each of the predefined object placements associated with a desired 
action, and for matching the recognised object placement with the corresponding desired action by 

s; and 



a user interface for coordinating with the user the performance of the desired action. 

21. A system for coordinating the placement of a plurality of interacting joint replacement 
components for reconstruction of a patient's joint, the system comprising: 

a tracking system for recording a set of reference points coupled to the patient and for monitoring 
placement of an instrument relative to the joint, the reference points being located on either side of 
title joint for indicating a reference geometry of the joint; 

a processor for coordinating the preferred placement of foe components in the reconstructed joint 
and for coordinating foe placement of the instrument, both of foe placements being based on the 
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predefined dimensions of the components and the reference geometry of the joint; and 

a display for displaying a feedback indicator to a user of the system, the feedback indicator for 
guiding the placement of the instrument relative to the joint; 

wherein a virtual representation of the components and the instrument are configured for display on 
the display with representative images of the joint 

22. The system of claim 21, wherein the reference geometry provided by the reference points is 
selected from the group comprising a reference length and a reference angle. 

23. The system of claim 22, wherein a feedback indicator is selected from die group comprising 
visual, auditory and other sensory indicators. 

24. The system of claim 23, wherein the reference geometry is represented by the display as the 
feedback indicator. 

25. The system of claim 22, wherein reference length is calculated as the distance between the pair 
of reference points. 

26. The system of claim 25, wherein the preferred placement of the components is coordinated by a 
calculation relative to the reference length and the dimensions of the components such that the 
reconstructed joint length resembles the reference length, 

27. The system of claim 26, wherein the reconstructed joint is selected from a group comprising a 
hip joint and a knee joint 

28. The system of claim 22, wherein the reference angle is calculated based on the reference points 
to represent the relative orientation between the patient's anatomy on either side of the joint 

29. The system of claim 28, wherein said preferred placement of the components is calculated with 
reference to the reference angle, the reference length and the dimensions of the components such feat 
the reconstructed joint orientation resembles the reference angle. - 

30. The system of claim 28, wherein in a hip replacement surgery, the reference angle is a femora] 
anteverston angle, measured between a plane defined by atrans-epicondylar axis and a long axis of a 
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femur and a vector of the femoral neck. 

31. The system of claim 23, wherein the feedback indicator is configured to guide the user's 
placement of the instrument selected ton flic group comprising: relative to an axis related to the 
preferred placement of the components] and relative to the reference geometry. 

32. The system of claim 3 1, wherein flie feedback indicator is configured to inform the user if the 
placement of fhe instrument is within a specified tolerance of the axis direction or the reference 
geometry. 

33. The system of claim 32, wherein the feedback indicator is auditory. 

34. The system of claim 23 wherein the feedback indicator is configured to inform the user of 
potential impingement of the replacement components. 

35. The system of claim 21, wherein a magnified version of fee virtual representation of the 
instrument is displayed on the display to correspond with the planned size of the replacement - 
component. 

36. A method for coordinatingfijepreferredplacementof apluraKty of interacting joint replacement 
components for reconstruction of a patient's joint, the method comprising fhe steps of: 

recording a pair of reference points coupled to the patient, the reference points being located on 
either side of the joint for indicating a reference geometry of the joint; 

determining the preferred placement of the components in the reconstructed joint and determining 
the placement of the instrument, both of the placements beingbased on predefined dimensions of Ihe 
components and the reference geometry of the joint; 

monitoring placement of an instrument relative to the joint; and 

displayingafeedback indicator to auser of the system, the feedback indicator guiding flieplacemefct 
of fhe instrument relative to the joint; 

wherein a virtual repra^ 

fhe user with representative images of the joint 
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37, A computer program product for coordinating the preferred placement of a plurality of 
interacting joint replacement components for reconstruction of a patient's joint, die program 
comprising: 

a computer readable medium; 

a digital imaging module stored on the medium for recording a pair of reference points coupled to 
the patient and for monitoring placement of an instrument relative to the joint, the reference points 
being located on either side of the joint for indicating a reference geometry of foe joint; 

a processing module stored on the medium for coordinating the preferred placement of the 
components in foe reconstructed joint and for determining foe placement of the instrument, both of 
foe placements being based on foe predefined dimensions of foe components and foe reference 
geometry of the joint; and 

a display module stored on foe medium for displaying a feedback indicator to a user of foe system, 
foe feedback indicator for guiding foe placement of the instrument relative to foe joint 

wherein a virtual representation of foe components and foe instrument are configured for display to 
foe user with representative images of foe joint 

38. A system for coordinating the preferred placement of a joint replacement component for 
reconstruction of a patient's joint, foe system comprising: 

a tracking system for recording at least three reference points of foe joint and for monitoring 
placement of an instrument relative to foe joint, foe reference points for indicating a reference 
surface geometry of foe joint, foe instrument configured for removing bone material relative to foe 
reference surface geometry; 

a processor for coordinating foe preferred placement of foe components in the reconstructed joint 
and for determining foe placement of foe instrument, both of foe placements being based on foe 
predefined dimensions of foe components and foe reference surface geometry of the joint; and 

a display for displaying a feedback indicator to a user of foe system, foe feedback ipdicator for 
guiding the placement of foe instrument relative to foe reference surface geometry of foe joint; 
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wherein a virtual tcpresentation of the component and the instrument are configured for display to 
the user with representative images of the joint, 

39. The system of claim 38, wherein the feedback indicator is selected from (he group comprising 
visual, auditory, tactile and other sensory indicators. 

40. The system of claim 39, wherein the reference surface geometry provided by the reference 
points defines a surface contour. 

41 . The system of claim 40, wherein the surface contour is used to define a portion of the joint that 
is to be removed by the instrument to enable the preferred placement of the joint replacement 
component. 

42. The system of claim 40, wherein die feedback indicator is configured to guide the user's 
placement of the instrument's depth relative to the surface contour. 

43. The system of claim 39, wherein the feedback indicator is configured to inform the user if the 
depth placement of the instrument is within a specified tolerance of a pre-defined depth relative to 
die reference surface geometry. 

44. The system of claim 43, wherein the feedback indicator is visual 

45. The system of claim 44, wherein the visual feedback indicator is a colour coded depth guide. 

46. The system of claim 40, wherein the surface contour is a plane associated with an interface 
surface of the joint 

47. A method for detennining the orientation and position of an axis defined by at least one 
attribute of an anatomical region relative to a reference frame, the method comprising the steps of: 

(i) tracking the position and orientation of the anatomical region relative to the frame; 

(n) tracking the position and orientation of an imaging device relative to the frame; 

(iii) aligning the imaging device with the axis based on the attribute; 

(iv) determining the position and orientation of the aligned imaging device relative to the 
position and orientation of the region; and, 
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